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METHOD AND APPARATUS FOR TIME-SHIFTING VIDEO AND TEXT IN A 
TEXT-ENHANCED TELEVISION PROGRAM 

5 

CROSS-REFERENCE TO RELATED APPLICATIONS 

This application claims the benefit of provisional patent application serial no. 
60/044,161, filed April 21,1997, and provisional patent applicalion serial no. 60/052,248, 
1 0 filed July 11,1 997, the disclosures of which are hereby fully incorporated by reference. 

BACKGROUND 

The present invention is a system wherein television program-related information 
J J (PRI) is embedded in the vertical blanking interval (VBI) of a television signal for display on 
a viewer's television screen at (he same lime as the television program. The PR! is typically 
contained in an Internet site, the addresses for v/hich are embedded in the television signal. 
The Internet site addresses may also be transmitted synchronously with, but separate from the 
20 video portion, e.g., via an interface device such as a telephone or cable modem. Typically 
such an ''enhajiced" television program consists of a display with the video portion of the 
program in a picture-in-picture (PIP) window and the PRl in the remaining portion of the 
display area of the television screen. 

This PRJ may be any textual or graphic infomiation associated with the current 
television program. The PRJ may consist of a textual display of a World Wide Web (WWW) 
or other Internet site address to which the viewer can choose to link to through a connection 
with an Internet Service Provider. Alternatively, the PRI may be contained in one or more 
Web pages, the addresses of which are inserted into the vertical blanking interval (VBI) and 
are automatically retrieved by tlie user's termmal and displayed on the display screen. Some 
examples of PRI are cast members' biographies, trivia about the show, information relating to 
n 5 the particular episode or scene, and closeups of infoiTnation that cannot be readily seen or is 
hidden in the video portion of the program. Some of the PRJ ma>' be time dependent on the 
program. For example, the PRI may change to correspond to a particular scene or frame of 
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the television program. 

With so much infonnation on the screen, some of which may be changing ai a fairl\ 

5 

rapid pace, it is desirable to provide the viewer the option of pausing a particular frame of a 
text-enhanced program display and then resume viewing the program without losing 
continuity of the video and PRJ portions of the program or program content, 

jQ According to one embodiment of the invention, a television system is provided which 

allows a viewer of a PRJ-enhanccd television program to pause the program at a particular 
frame, examine the PRI at his or her leisure, perhaps browse through other linked Web 
pages, and then resume viewing the program from that frame, without losing continuity of 

1 5 the video and PRI portions of the program or program content. 

SUMMARY 

According to one embodiment of the invention, an apparatus for time-shifting video 
and program related information (PRI) in an enhanced television program is provided which 
includes a display screen, a tuner for receiving a television signal with embedded data 
representative of an address for an Internet site including PRJ, means for extracting the 
embedded data from the television signal, a memory for storing the embedded data, input 

25 

means for inputting viewer commands, a time-shifting apparatus capable of simultaneously 
storing the television signal as it is received and outputting the stored television signal for 
display, means for communicating with an internet service provider to retrieve infonnation 
from the Intemet site including the PRK and a microcontroller. The microcontroller is 
configured to retrieve the Intemet site address from memory and retrieve the PR] from the 
Internet site in response to a first viewer command, generate a composite display including a 
television program contained in the television signal in a first portion of the display and the 
35 PRI in a second portion of the display in response to the first viewer command, control the 
time-shifting apparatus to store the television signal as it is received and display a still frame 
from the stored television signal in a first portion of the display screen in response to a 
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second viewer command, and control the lime-shifting apparatus to output the portion of the 
stored television signal subsequent to the still frame for display in the first portion of the 

5 

display in response to a third viewer command. 

According to another embodiment, a method for time-shifting video and program 
related informalion (PRJ) in an enhanced television program is provided which includes the 
IQ steps of receiving a television signal with embedded data representative of an address for an 
Inlemel site including PRJ, extracting the embedded data from the television signal, storing 
the embedded data in a memoiy, selecting an Internet mode in response to a first viewer 
command, communicating with an Internet service provider to retrieve information from the 
1 5 Internet site including the PRl, displaying a television program contained in the television 
signal in a first portion of a display screen and the PRl in a second portion of the display 
screen, storing the television signal in a time-shifting apparatus and continuously displaying a 
still frame from the stored television signal in response to a second viewer command, and 
simultaneously displaying the television program subsequent to the still frame from the 
stored television signal and continuing to store the television signal as it is received in 
response to a third viewer command. 

According to an altemate embodiment, one or more suspend flags are embedded in the 
television signal, and the "pause" operation wherein the television signal is stored in the time- 
shifting apparatus and the still frame displayed in response to detection of such a suspend 
flag. This feature may be deactivated such that the "pause" operation is only perfonned in 
response to a viewer command. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The foregoing features and advantages of the invention will be better understood by 
35 refeiring to the following drawings: 

FIG. I is a schematic block diagram of a time-shifting apparatus according to one 
embodiment of the invention; 
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FIG. 2 is a display screen in an Internet mode of the lime-shifting apparatus; 

FIG. 3 is a display screen accessed by the viewer from the display screen of FIG. 2: 

and 

FIG. 4 is an alternate layout of the display screens of FIGS. 2 and 3. 

DETAILED DESCRIPTION 

In FIG. L the reference numerals refer to the same elements described in application 
Serial No. 08/475,395 filed on June 6. 1995, the disclosure of which is incorporated fully 
herein by reference. In addition, the system includes 1) an Internet service provider 33 
connected to microprocessor 24 by a transmission link 34 such as a telephone network or a 
television cable. 2) a VBI decoder 35, 3) a website data memor\' 36 (memory 36 could be 
part of the RAM of microprocessor 24 or in tenns of the disclosure of the '395 application, 
memoiy 22), and 4)a digital Storage Device 52 with associate analog-to-digital and digitai- 
to-analog converters 50, 54. An interface device such as a telephone or cable modem (not 
shown) couples transmission link 34 to microprocessor 24, if necessary. Internet service 
provider 33 is connected to an Internet backbone in well known fashion to access data at any 
site on the WWW. 

Storage device 52 is a television signal time-shifting apparatus. One such time- 
shifting apparauis is disclosed in U.S. patent application Serial No. 08/388.345 to Russo. ei 
al. filed February 14. 1995, which is fully incorporated herein. Such a time-shifting 
apparatus includes an optical disc for storage of video programs and separate READ and 
WRITE heads which operate simultaneously such that real time program information can be 
stored on the disc while previously stored information on the disc can be read and output to 
the television signal for display. Other storage media which are capable of rapidly storing 
extremely large amounts of information may also be used, including magnetic tape, optical 
disk, magneto-optical disk, or solid stale memory (i.e., a high capacity charge coupled 
device), video RAM, etc. 
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The Storage Device 52 holds a large video data buffer (not shown) for storing the 
television program in digital fomi. Preferably, the Storage Device is a random access storage 
medium allowing concurrent reading and writing operations, so that the incoming television 
signal data may be written to the Storage Device while earlier stored television signal data is 
being read out for display on TV 20 (that is, time-shifting of the television signal data is 

I Q performed). The Storage Device 52 has two heads that are separately positionable. When 
display of the television program is to be suspended, the read head is kept in the same 
position until a resume command is received. The write head, however, keeps moving to 
record tlie incoming television signal data. 

15 To enable a television viewer to access information about a television program that the 

viewer is watching, PRI is embedded in the VBl of the television signal carrying the 
television program. For example, the PRI may be textual information regarding actors and 
actresses in the show, advertisements of program-related merchandise, brief descriptions of 
the plot of future episodes ot the television program, or any other text regarding the television 
program, or the PRI may be text representing web pages containing such infoiTnation. 

According to a preferred embodiment, the PRJ is contained on a web page, the address 
for which is embedded in the VBl of the television signal. When the television signal 

25 

canning the program being watched is captured by tuner 1 1 , the website data embedded in its 
VBl is stripped out by VBl decoder 35 and sent to microprocessor 24 for storage in website 
data memor)' 36. The memory addresses of the website names are linked to the website 

2Q addresses in memory 36. An icon appears on the screen of television 20 when the television 
program is displayed ftill screen, i.e., in the TV mode, to inform the viewer that website data 
accompanies the television signal and is stored in memory 22. If the viewer wishes to access 
a website in connection with the television program, the viewer presses a button on a viewer 

35 input device 28 such as a remote controller, which introduces the Inleniet mode of operation 
shown in FIG. 2 and described below. Microproce.s.sor 24 is programmed to carry out this 
operation. By repeatedly pressing a button on the viewer input device, the viewer can toggle 
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back and forth between the TV mode and the Internet mode. Ahhough viewer inputs are 
discussed herein as initiated by buttons on a remote controller, other input devices can also be 
used. For example, a cursor could be displayed on the television screen which is movable 
around the screen and a selection can be made (i.e., "clicked") when the cursor is in a desired 
location of the screen. 

In the hitemet mode, the video portion of the television program last viewed in the TV 
mode is displayed in area 42. As an option, a textual description ofliic program is displayed 
in an area 44 and information about the television program, i.e., program title, station name, 
and channel number are displayed in a banner 49 underneath areas 42 and 44. A message is 
1 5 displayed at the top of an area 46 to prompt the viewer to select from a number of website 
names displayed in area 46 by moving a cursor 48 with anow keys on the viewer input 
device. For example, if the television program is a serial television show, for example, 
"Manied With Children," the website names could be information related to the show. After 
a website name is selected, the viewer presses a button on the viewer input device. As a 
result, the website address to which the selected website name is linked is retrieved from 
memor)' 36 by microprocessor 24 and sent through the telephone or cable interface to 
Internet service provider 33. (If desired, this function of microprocessor 24 could be carried 
out by commercial equipment sold under the trademark WFB TV.) The information at the 
addressed website is downloaded from Internet service provider 33 over link 34 to 
microprocessor 24 and then displayed on the screen simultaneously with the television 
30 program to which the infoniiation relates, as illustrated in FIG. 3, after being composed by 
video processor 30. As illustrated in FIG. 3, the name of the website can be displayed above 
the text of the information from the website. If the television program is a serial television 
show, as previously stated, the displayed information could include information about the 
35 episode, cast biographies, and trivia related to the show. Fhe viewer then navigates about the 
website in the manner dictated by the viewer's software to find the desired information. 
An alternative layout of the television screen for the present invention is shown in 
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FIG. 4. In this layout, the television program is displayed in a majority portion 60 of the 
screen while the PRl is displayed along two border regions 62. The Web page including the 

5 

PRl can be speciikally configured to display the PRI in the border regions and an area 
designated for the real time image in the PIP. 'Flie PIP circuitry 19 is specially configured to 
a smaller reduction ratio, for example 1 .5: 1 rather than 3:1 for a standard PIP, in order to 
J Q produce the larger PIP display. 

The screen portion 60 displays a television program consisting of moving images. 

Referring back to FIG. 1, when the viewer interacts with the website data or other PRJ 
displayed on tlie television screen, the viewer's attention is diverted from the television 
1 5 program being shown to the website data. The viewer is then missing what is happening in 
the television program until the viewer's interaction with the PR] is concluded. To overcome 
this situation, an additional component. Storage Device 52, described above, is added to the 
system to "time-shift" the display of the television program. As the television signal is being 

20 

received by Tuner 1 1 , the signal is forwarded through IF Amp 1 2 and Picture DET 1 3 to 
VCR 1 7. The VCR sends the signal through an analog to digital converter (A/D) 50 to 
Storage Device 52. The Storage Device is under the control of the Microprocessor 24 and is 
capable of storing the incoming television signal in real-time as digital infonnation for future 

25 

use. 

As the television signal is being stored, if a viewer wants to interact with the PRl such 
as website data or other textual infonnation being displayed on the television screen, the 

2Q viewer sends a command to the microprocessor 24 via the viewer input device 28. The 

viewer action to send the command could, for example, consist of pushing a button on the 
viewer input device. In response, the microprocessor 24 controls VCR 1 7 to output the 
television signal to the Storage Device 52 which begins storing the television signal. 

35 including the PRI information embedded in the VBI. The Storage Device 52 simultaneously 
outputs the first stored frame of the video signal to the signal processing unit for extended 
display on television 32. The television 32 continues to display this frame until controlled by 
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the viewer to continue without effect on any viewer activity with the PHI shown in the 
remainder of the display screen. The viewer then internets with the PRJ as described abo\ e. 

5 

When the viewer is done interacting with the PRK the viewer sends a command to the 
microprocessor 24 to resume display of the television program. However, instead of 
displaying the incoming television signal from Tuner 1 1 , the VCR directs the deliveiy of the 
jQ stored television signal data output from the READ head of Storage Device 52 through 

Digital-to-Analog Converter (D/A) 54 and SW 1 8 to PIP 1 9 for display on TV 20. The data 
displayed is that part of the television program immediately subsequent to the point of 
suspension. That is, it has been time-shifted. The incoming television signal data continues 
1 5 to be stored by the WRJTE head of the Storage Device 52 in a time-ordered manner 
regardless of the functioning of the READ head. In other words, when display of the 
television program is to be suspended, the READ head is kept in the same position until a 
resume command is received. The WRITE head, however, keeps moving to record the 
incoming television signal data. Hence, at this time the data being stored is not the same data 
that is being displayed; there is a time lag between the two sets ofdata. In this manner, the 
viewer may continue watching the program without losing continuity of the program or PRl 
content. The viewer can position a cursor and enter input to freeze the display of the 
television program image on command. 

In another embodiment, the PRJ is contained on several web pages, each 
corresponding to a particular portion of the program and transmitted chronologically 
2Q throughout the duration of the program. The television signal including the embedded 

website addresses is stored on the Storage Device 52 and hence the corresponding Web pages 
remain linked to the appropriate portion of the television program as the signal is read out 
from the Storage Device 52 after a "pause" operation. In this manner, the viewer may 
35 continue watching the program without losing continuity of the program or PR] content. 

The audio portion of the television signal is also stored in the Storage Device along 
with the video portion. When the live television program is put into a "freeze'' frame state. 
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the audio portion of the television program is also suspended and not transmitted to the 
viewer. Instead, the audio portion is stored. When display of the suspended television 

5 

program is resumed, the audio data is obtained from the Storage Device along with the video 
portion and forwarded by VCR 17 through Sound Amp 15 and loudspeaker 16 to the viewer. 
In another embodiment, the television signal is already in digital fonri, such as for 
J Q high-definition digital television (HDTV). Therefore, an analog to digital conversion is 

unnecessary. The Storage Device continuously stores the television signal in a wraparoimd 
fashion whenever the system is operational, overwriting the oldest previously stored 
television signal data when the Storage Device becomes full. The Storage Device should be 
1 5 large enough to hold two to three hours of television programming before overwriting earlier 
broadcast television signal data. 

The present invention allows a viewer to inteiTupt his or her viewing of a television 
program to interact with PRl carried in the VBI of the television signal and displayed on the 
television screen, and yet rejoin the television program at a later point in time without 
missing any of the program. 

In another embodiment, tlie display of the incoming television signal may be 
suspended automatically by inserting a suspend flag into the VBL rather than by an explicit 
action by the viewer. When the suspend flag is detected by the microprocessor 24 after 
decoding by VBI decoder 35, the microprocessor instructs the VCR via a control link (not 
shown) to suspend the current display of the television signal. Resumption of display of the 
television program is commenced by viewer input. The viewer could also override the 
automatic suspension feature provided by the suspend flag by setting a predetermined control 
value to override all automatic suspend flags, or by entering viewer input when the 
suspension activity occurs in order to rapidly rejoin the television program in progress. 
35 Alternatively, a resume flag is inserted into the VBI at a predetermined time after the suspend 
flag. When the rcsume flag is received, the microprocessor automatically controls the VCR 
to resume display of the television program. 
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According to yet another embodiment, the viewer may access the information as any 
otlier storage media, such as a video tape, and pause, rewind, or fast forward to different 

5 

portions of the program stored on the disc after the initially "pause" command. It may be 
desirable to continuously record the program on the time-shifting apparatus 46 regardless of a 
viewer "pause'' command to allow for these functions over a period of time, limited only by 

I Q the storage capacity of the time-shifting device. 

Although the present invention has been described with respect to particular 
embodiments, those skilled in the art will appreciate that the present invention may be 
modified without departing from the scope of the invention. Accordingly, all such 

1 5 modi fications are intended to be included within the scope of the invention as defined by the 
following claims. 

20 
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CLAIMS: 

1 . Apparatus for time-shifting video and program related infonnation (PRJ) in an 

5 

enhanced television program comprising: 
a display screen; 

a tuner for receiving a television signal with embedded data representative of an 
address for an hiternet site including PRI; 

means for extracting the embedded data from the television signal; 

a memory for storing the embedded data; 

input means for inputting viewer commands; 
1 5 a time-shifting apparatus capable of simultaneously storing the television signal as it is 

received and outputting the stored television signal for display; 

means for communicating with an internet service provider to retrieve information 
from the internet site including the PRl; and 

20 

a microcontroller comprising 

means for retrieving the internet site address from memory and retrieving the 
PRJ from the internet site in response to a first viewer command; 

means for generating a composite display including a television program 

25 

contained in the television signal in a first portion of the display and the PRJ in a 
second portion of the display in response to the first viewer command, 

means for controlling the time-shifting apparatus to store the television signal 
as it is received and display a still frame from the stored television signal in a first 
portion of the display screen in response to a second viewer command, and 

means for controlling the lime-shifting apparatus to output the portion of the 
stored television signal subsequent to the still frame for display in the first portion of 
35 the display in response to a third viewer command. 
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2. The apparatus of claim 1 wherein the time-shifting apparatus is an optical disc. 

5 

3. Tlie apparatus of claim 1 wherein the time-shifting apparatus is a solid state 
memory. 

J Q 4. The apparatus of claim 1 wherein the PRI comprises text and graphics related to 

the television program. 

5. The apparatus of claim 4 wherein the PRI comprises a plurality of website 
1 5 addresses. 

6. The apparatus of claim 5 wherein the microcontroller comprises means tor 
retrieving infonriation from one of said plurality of website addresses in response to a fourth 
viewer command. 

7. The apparatus of claim I wherein the first portion of the display covers a minor 
portion of the display screen and the second portion of the display covers a major portion of 

25 

the display screen. 

8. The apparatus of claim I wherein the first portion of tlie display covers a major 
portion of the display screen and the second portion of the display covers a minor portion of 
the display screen. 

9. The apparatus of claim 1 wherein the means for communicating with the 
35 internet service provider is a modem. 
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1 0. A method for time-shifting video and program related information (PRI) in an 
enhanced television program comprising the steps of: 

5 

receiving a television signal wilh embedded data representative of an address for an 
internet side including PRI; 

extracting the embedded data from the television signal; 
^ Q storing the embedded data in a memory; 

selecting an internet mode in response to a first viewer command; 

communicating v^ith an internet service provider to retrieve information from the 
internet site including the PRJ; 
1 5 displaying a television program contained in the television signal in a first portion of a 

display screen and the PRI in a second portion of the display screen; 

storing the television signal in a time-shifting apparatus and continuously displaying a 
still frame from the stored television signal in response to a second viewer command; and 

20 

simultaneously displaying the television program subsequent to the still frame from 
the stored television signal and continuing to store the television signal as it is received in 
response to a third viewer command. 

25 

1 1 . Tlie method of claim 10 wherein the PRI comprises a plurality of website 
addresses and ftirther comprising the steps of: 

selecting one of the plurality of website addresses in the PRI; 
^ retrieving infomiation from the selected website address; and 

displaying the information from the selected website address. 



1 2. A method for time-shifting video and program related information (PRI) in an 
35 enhanced television program comprising the steps of: 

receiving a television signal witli embedded data representative of an address for an 
internet side including PRI and a suspend flag; 
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extracling the embedded data from the television signal; 
storing the embedded data in a memory; 

5 

selecting an internet mode in response to a first viewer command; 

communicating with an internet service provider to retrieve infomiation from the 
internet site including the PRI; 
^ Q displaying a television program contained in the television signal in a first portion of a 

display screen and the PR] in a second portion of the display screen; 

detecting the suspend flag; 

storing the television signal in a time-shifting apparatus and continuously displaying a 
1 5 still frame from the stored television signal; and 

simultaneously displaying the television program subsequent to the still frame from 
the stored television signal and continuing to store the television signal as it is received in 
response to a second viewer command. 

20 

13. The method of claim 12 wherein the step of storing ten television signal in the 
time-shifting apparatus is performed in response to detecting the suspend flag. 

25 

14. The method of claim 12 wherein the step of storing the television signal in the 
time-shifting apparatus is performed in response to a third viewer command. 

30 
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